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DETAILED ACTION 

Restriction to one of the following inventions is required under 35 U.S.C. 121: 

I. Claims 1-10,15-20, 30-40, and 44-47 are drawn to video segment editing 
classified in class 345, subclass 723. 

II. Claims 25-29, and 48-49 are drawn to video traversal control classified in 
class 345, subclass 720. 

III. Claims 11-13, 21-24, 41-42, 50-51 are drawn video interface classified in 
class 345, subclass 719. 

The inventions are distinct, each from the other because of the following reasons: 

Inventions group 1 and group 2 are related as combination and subcombination. 
Inventions in this relationship are distinct if it can be shown that (1 ) the combination as 
claimed does not require the particulars of the subcombination as claimed for 
patentability, and (2) that the subcombination has utility by itself or in other 
combinations (MPEP § 806.05(c)). In the instant case, the combination as claimed 
does not require the particulars of the subcombination as claimed because has a 
different mode of operation for scene selector that searches through a database and 
organizes the scenes. The subcombination has separate utility such as a different 
mode of operation to display the scenes of a playlist, and allows user feedback 
commands to update the playlist. 

Inventions group 1 and group 3 are related as combination and subcombination. 
Inventions in this relationship are distinct if it can be shown that (1 ) the combination as 
claimed does not require the particulars of the subcombination as claimed for 
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patentability, and (2) that the subcombination has utility by itself or in other 
combinations (MPEP § 806.05(c)). In the instant case, the combination as claimed 
does not require the particulars of the subcombination as claimed because has a 
different mode of operation for scene selector that searches through a database and 
organizes the scenes. The subcombination has separate utility such as a different 
mode of operation for retrieving scenes and identifying the relationship to create a 
playlist. 

Inventions group 2 and group 3 are related as combination and subcombination. 
Inventions in this relationship are distinct if it can be shown that (1 ) the combination as 
claimed does not require the particulars of the subcombination as claimed for 
patentability, and (2) that the subcombination has utility by itself or in other 
combinations (MPEP § 806.05(c)). In the instant case, the combination as claimed 
does not require the particulars of the subcombination as claimed because a different 
mode of operation to display the scenes of a playlist, and allows user feedback 
commands to update the playlist. The subcombination has separate utility such as a 
different mode of operation for retrieving scenes and identifying the relationship to 
create a playlist. 

Because these inventions are distinct for the reasons given above and have 
acquired a separate status in the art because of their recognized divergent subject 
matter, restriction for examination purposes as indicated is proper. 

During a telephone conversation with Applicants' Attorney, Himanshu S. Amin on 
December 1 , 2004 a provisional election was made without traverse to prosecute the 
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invention of group 1 claims 1-10,15-20, 30-40, and 44-47. Affirmation of this election 
must be made by applicant in replying to this Office action. Claims 25-29, and 48-49, 
group 2, and 11-13, 21-24, 41-42, 50-51 group 3, withdrawn from further consideration 
by the examiner, 37 CFR 1 .142(b), as being drawn to a non-elected invention. 

Applicant is reminded that upon the cancellation of claims to a non-elected 
invention, the inventorship must be amended in compliance with 37 CFR 1 .48(b) if one 
or more of the currently named inventors is no longer an inventor of at least one claim 
remaining in the application. Any amendment of inventorship must be accompanied by 
a request under 37 CFR 1 .48(b) and by the fee required under 37 CFR 1 .17(i). 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 45-47 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

In claims 45, 46, & 47 are only data structure embodied on a "memory readable 
media" (hereafter, computer readable medium), which does not serve a particular 
function, nor provide functionality to obtain any type of recited utility, namely the 
"executable instructions" recited in the preamble. Further, the computer readable 
medium containing the data structure(s) do not fall within one of the five categories of 
statutory subject matter, namely, new and useful process, machine, manufacture, 
composition of matter, or any new and useful improvement thereof. The claim(s) are 
directed towards a data structure per se, which is non-statutory in nature. See MPEP 
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§2106(IV)(B)(1). When nonfunctional descriptive material is recorded on some 
computer-readable medium, it is not statutory since no requisite functionality is present 
to satisfy the practical application requirement. . 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

1. Claims 1-6, 8-10, 43, 44 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Jain et al (U.S. Patent Number 6,567,980). 

2. In regards to claim 1 , Jain discloses a system that facilitates non-linear viewing of 
media, the system comprising: 

a scene selector that scans a digitized media and selects a scene in the digitized 
media (col. 2 lines 10-19) [The cataloger transforms video into a 
powerful data type that is both compelling and profitable in 
both Web and client-server environments. Using advanced media 
analysis algorithms that automatically watch, listen to and read 
a video stream, the multimedia cataloger intelligently extracts 
metadata-keyframes , time codes, textual information and an audio 
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profile from the video in real-time .]. This means that the system is 
selecting scenes of importance by user preference; 

a metadata generator that produces metadata associated with the selected 
scene and relates the metadata to the selected scene (col. 2 lines 6-9) [in one 

aspect of the invention, there is a media cataloging and media 
analysis application which performs real-time, or non-real-time, 
indexing and distribution of video across an enterprise .]; and 

an organizer that places the selected scene and the metadata in a media store to 
facilitate non-linear viewing of one or more scenes (col. 2 lines 20-23) [This 
information becomes the foundation of a rich, frame-accurate 
index that provides immediate, non -l inea r access to any segment 
of the video.]; 

3. In regards to claim 2, Jain states the scene selector selects a scene based on at 
least one of, face recognition, item recognition, voice recognition, color recognition, 
mood recognition and theme recognition (col. 2 lines 14-19) [Using advanced media 
analysis algorithms that automatically watch, listen to and read 
a video stream, the multimedia cataloger intelligently extracts 
metadata-keyframes, time codes, textual information and an audio 
profile from the video in real-time.]. 

4. In regards to claim 3, Jain discloses the scene selector selects a scene based, at 
least in part, on an input from a user (col. 8 lines 42-46) [A set of output filters 
484 are installed during system start-up. These registration 
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processes, as well as user input and output functions 550, 554, 
are further described in conjunction with FIG. 11 below.]. 
5. In regards to claim 4, Jain states at least one of the face recognition, item 
recognition, voice recognition, color recognition, mood recognition and theme 
recognition is adapted by a machine learning technique based, at least in part, on the 
input from the user (Table 1 col. 7; col. 7 lines 7-12) [ Audio Class Enumerated 
Speech , Silence, Music, Applause, Track Classes Siren, etc..., 
each spanning a time interval when that classification was valid 
Speech Track Text fragment Each text fragment spans a time 
interval Keyword Track Word (text) keyword utterance spans a 
short (1/2 sec) time interval Speaker ID Enumerated 
Identifiers of individuals whose Track Classes speech is 
recognized. . . each Speaker ID spans a time interval when that 
speaker was speaking Clip Track Label Set (user Different Label 
Set schemas can be defined set of used in different 
applications .] [That is, unique metadata can be defined and added 
to the Video Cataloger 110 by a user . Custom metadata tracks 
could include information provided in collateral data to the 
video information . ]. The examiner interprets that these metadata tags allows the 
user to search digital videos, which are inputted by the user. Also, the user is allows to 
create custom tags for quicker searches, and creating a custom tags allows the 
computer to adapt a new learning technique. 
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6. In regards to claim 5, Jain discloses the metadata generator produces at least 
one of, a date, a time, a length, a subject, a mood, a theme, a color, a person name, a 
set of person names, an item name and a set of item names associated with the scene 
(col. 6 lines 2-5; col. ) [The Vidsync daemons also are responsible for 
returning certain pieces of information to the Cataloger, such 
as the actual start time, and a digital video asset ID or name 
for later use.]. 

7. In regards to claim 6, Jain states method that facilitates non-linear viewing of 
media, comprising: 

selecting a scene from a set of digitized media (col. 6 lines 18-23 & col. 4 lines 
38-40) [When a piece of metadata (such as a keyframe) is used to 
index the digital video, an absolute time from the beginning of 
the digital video is computed by adding the delta-time to the 
time-code of the metadata .]& [Finally, at the bottom of the 
window 170 is a timeline 180 that depicts the total time of the 
capture session, with a highlighted section corresponding to the 
currently selected range of keyframes]; 

annotating the selected scene with metadata (col. 7 lines 28-32; col. 8 lines 60- 
64) [During metadata capture, the user may mark video clips and 
annotate them. This input 552 is captured by the GUI Input 
Capture element 550.] [ User defined annotations may also exist 
here. Each of the metadata tracks is a collection of data 
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objects 406, 408, 410, 412, etc. that hold the metadata for a 
specific feature extractor,]. This allows the user to writes a description of a 
keyframe, or selected a metadata with a description; and 

storing the annotated scene to facilitate non-linear retrieval of the annotated 
scene (col. 2 lines 20-23) [This information becomes the foundation of a 
rich, frame-accurate index that provides immediate, non-linear 
access to any segment of the video.]. 

8. In regards to claim 8, Jain states selecting the scene from the set of digitized 
media comprises: 

automatically scanning one or more scenes from the set of digitized media 
recognition (col. 2 lines 15-20) [Using advanced media analysis algorithms 
that automatically watch, listen to and read a video stream , the 
multimedia cataloger intelligently extracts metadata-keyframes, 
time codes, textual information and an audio profile from the 
video in real-time.]. The examiner interprets that automatically watching and 
listening is done by the system; and 

automatically selecting the scene based on at least one of face recognition, item 
recognition, voice recognition, color recognition, mood recognition and theme 
recognition (col. 2 lines 15-20) [Using advanced media analysis algorithms 
that automatically watch , listen to and read a video stream, the 
multimedia cataloger intelligently extracts metadata- keyframes , 
time codes, textual information and an audio profile from the 
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video in real-time .]. The examiner interprets by that the system after 
automatically listening and watching also automatically selects keyframes. 

9. In regards to claim 9, Jain discloses annotating the selected scene with metadata 
comprises associating at least one of, a date, a time, a length, a subject, a mood, a 
theme, a color, a person, a set of people, an item and a set of items with the selected 
scene (col. 6 lines 2-5; col. 8 lines 60-63 ) [The Vidsync daemons also are 
responsible for returning certain pieces of information to the 
Cataloger, such as the actual start time, and a digital video 
asset ID or name for later use.] [During metadata capture, the 
user may mark video clips and annotate them. This input 552 is 
captured by the GUI Input Capture element 550.]. 

10. In regards to claim 10, Jain states storing the annotated scene to facilitate non- 
linear retrieval of the annotated scene comprises storing the annotated scene in at least 
one of, a database and a datacube (col. 3 lines 60-67) [Metadata Server 13 o : may 
be as simple as a file system containing hypertext markup 
language (HTML) files, or as complex as a relational database 
supporting a client-server application environment for media 
management .]. 

11. In regards to claim 43, Jain discloses a data structure that facilitates non-linear 
viewing of media items, the data structure comprising: 

a first field that holds a media item (Figure 2 item 176); and 
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a second field that holds a metadata item related to the media item, where the 
metadata facilitates at least one of identifying the media item, locating the media item 
and locating a related media item (Figure 2 item 178). 

12. In regards to claim 44, Jain states in a computer system having a graphical user 
interface that comprises a display and a selection device, a method of providing and 
selecting from a set of graphical user interface elements on the display, the method 
comprising: 

retrieving a set of graphical user interface elements, each of the interface 
elements representing an action associated with facilitating the non-linear display of 
media items (col. 2 lines 20-24) [This information becomes the foundation 
of a rich, frame-accurate index that provides immediate, non- 
linear access to any segment of the video. ]; 

displaying the set of interface elements on the display (Figure 2); 

receiving an interface element selection signal indicative of the selection device 
selecting a selected entry from the set of interface elements (col. 4 lines 26-28) [a 
panel 172 displays the live video being digitized, with play, 
stop, etc. controls that interact remotely with the analog 
source via a deck controller 240 (FIG. 3) .] ; and 

in response to the interface element selection signal, initiating processing to 
facilitate non-linear viewing of media based, at least in part, upon stored metadata (col. 
4 lines 33-35) [panel 184 displays the user-defined clip annotations, 
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created by marking in- and out -points. The columns 186 and 188 
display the in- and out-time codes for the marked clip,] . 

13. Claims 45-47 are rejected under 35 U.S.C. 102(e) as being anticipated by Morris 
(U.S. Patent Application: 2002/0088000). 

14. In regards to claim 45, Morris states a data packet adapted to be transmitted 
between two or more computer components that facilitate annotating a media and 
facilitate the non-linear viewing of the media, the data packet comprising: 

a first field that stores a clip identifier that identifies a portion of a media (Figure 2 
item 52). The examiner interprets that the header of a data packet is used to identify the 
that this is a new packet, and the packet contains new information of the media; 
a second field that stores a metadata key that identifies an annotating metadata 
associated with the clip identified by the clip identifier (Figure 2 items 60 & 62; 
paragraph 20 ) [The metadata for the images is preferably stored 
within the image file 50 in individual image tags 60, each of 
which store various types of data that correspond and relate to 
particular captured image data 54.]; and 
a third field that stores data associated with the clip identified by the clip identifier 
(Figure 2 item 54). This is the field where the actual data is stored. 

1 5. In regards to claim 46, Morris states a memory for storing data for access by a 
computer component, the memory comprising: 

a data structure stored in the memory, the data structure holding: 
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a first field that stores a clip identifier that identifies a portion of a media (Figure 2 
item 52). The examiner interprets that the header of a data packet is used to identify the 
that this is a new packet, and the packet contains new information of the media; 

a second field that stores a metadata key that identifies an annotating metadata 
associated with the clip identified by the dip identifier (Figure 2 items 60 & 62; 
paragraph 20 ) [The metadata for the images is preferably stored 
within the image file 50 in individual image tags 60, each of 
which store various types of data that correspond and relate to 
particular captured image data 54.]; and 

a third field that stores data associated with the clip identified by the clip identifier 
(Figure 2 item 54). This is the field where the actual data is stored. 
16. In regards to claim 47, Morris states data packet adapted to be transmitted 
between two or more computer components that facilitate the non-linear viewing of a 
media, the data packet comprising: 

a first field that stores a clip identifier that identifies a portion of a media(Figure 2 
item 52). The examiner interprets that the header of a data packet is used to identify the 
that this is a new packet, and the packet contains new information of the media; 

a second field that stores a requested user action concerning the portion 
identified by the clip identifier (Figure 2 items 60 & 62; paragraph 20 ) [The metadata 
for the images is preferably stored within the image file 50 in 
individual image tags 60, each of which store various types of 
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data that correspond and relate to particular captured image 
data 54 .]; and 

a third field that stores metadata associated with the portion identified by the clip 
identifier, where the metadata is employed to adapt one or more clips according to the 
requested user action(Figure 2 item 54; paragraph 19) [image data 54 contains 
actual captured image data, the resolution of which depends on 
the settings of the camera 14. Image data 54 is typically 
stored in JPEG format, but may exist in whichever format is 
appropriate for the current location of image file 50 within the 
image processing chain of the camera 110 .]. This is the field where the 
actual data is stored. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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17. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jain et al 
(U.S. Patent Number 6,567,980) in view of Duncombe et al (U.S. Patent Number 
6,813,745). 

18. In regards to claim 7, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. 

The difference between the claims and Jain is the claims recite "manually 
scanning one or more scenes from the set of digitized media; and manually selecting 
the scene." 

Duncombe teaches a media system for storing electrical files that allow the user 
to enter search descriptions for digital retrieval similar to that of Jain. In addition, 
Duncombe discloses "manually scanning one or more scenes from the set of digitized 
media; and manually selecting the scene." (col. 3 lines 13-14) [fig. 8 is a flow 
diagram showing the steps taken by a user to select and view a 
plurality of suitable media clips.]. 

It would have been obvious to one of ordinary skill in the art, having the 
teachings of Jain and Duncombe before him at the time the invention was made, to 
modify the viewing of GUI taught by Jain to include the user to manual search media file 
of Duncombe, in order to obtain a system that does automatically or manually allows the 
user to search a digital media storage. 



Application/Control Number: 10/055,538 Page 16 

Art Unit: 2173 

One would have been motivated to make such a combination because it create a 
dynamic search engine for viewing digital media information would have been obtained, 
as taught by Duncombe. 

19. Claims 15-20, 30-40 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Jain et al (U.S. Patent Number 6,567,980) in view of Duncombe et al (U.S. Patent 
Number 6,792,573). 

20. In regards to claim 15, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. In addition, Jain teaches a scene retriever that 
retrieves one or more scenes and one or more pieces of annotating metadata 
associated with the one or more scenes from a media store (col. 2 lines 10-19) [The 
cataloger transforms video into a powerful data type that is 
both compelling and profitable in both Web and client-server 
environments. Using advanced media analysis algorithms that 
automatically watch, listen to and read a video stream, the 
multimedia cataloger intelligently extracts metadata-keyframes, 
time codes, textual information and an audio profile from the 
video in real-time .]; and a metadata analyzer that analyzes the one or more 
pieces of annotating metadata to identify one or more relationships between the one or 
more pieces of annotating metadata a metadata generator that produces metadata 
associated with the selected scene and relates the metadata to the selected scene (col. 
2 lines 6-9 & col. 2 lines 27-30) [In one aspect of the invention, there is 
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a media cataloging and media analysis application which performs 
real-time, or non-real-time, indexing and distribution of video 
across an enterprise .] [Synchronized encoding and indexing allows 
users to intelligently navigate through the video by using the 
index to go directly to the exact point of interest, rather than 
streaming it from start to finish.] 

The difference between the claims and Jain is the claims recite "a playlist 
generator that evaluates the one or more relationships and produces a playlist of related 
scenes, one or more viewers for viewing a scene listed in the playlist, one or more 
feedback receivers for receiving a feedback concerning the viewed scene; and a playlist 
updater for updating the playlist based, at least in part on the feedback." 

Duncombe teaches a system that allows the user to browser through video clips 
by receiving user feedback, which is similar to that of Jain. In addition, Duncombe 
teaches a playlist generator that evaluates the one or more relationships and produces 
a playlist of related scenes (col. 2 lines 34-36) [The method includes the steps 
of first organizing and formatting a plurality of media clips,], 
and one or more viewers for viewing a scene listed in the playlist (col. 2 lines 35-36) 
[and then playing the plurality of media clips based upon input 
supplied by the user .], and one or more feedback receivers for receiving a 
feedback concerning the viewed scene (col. 2 lines 50-55) [Once the plurality of 
suitable media clips have been played, the media system uses a 
means for soliciting feedback and a means for accepting feedback 
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from the user, and a means for reselecting the plurality of 
suitable media clips based upon the feedback of the user. ]; and a playlist updater for 
updating the playlist based, at least in part on the feedback (col. 2 lines 65-67) [A 
further objective is to provide a media system that solicits 
feedback from the user and modifies the media presentation in 
response to the feedback.]. 

It would have been obvious to one of ordinary skill in the art, having the 
teachings of Jain and Duncombe before him at the time the invention was made, to 
modify the viewing of GUI taught by Jain to include the user to preference search media 
file with a feedback control of Duncombe, in order to obtain a system that has the ability 
to generate a set of digital media clips in condition of user feedback. 

One would have been motivated to make such a combination because it create a 
dynamic search engine for viewing digital media information with feedback would have 
been obtained, as taught by Duncombe. 

21. In regards to claim 16, Jain in view of Duncombe discloses the one or more 
viewers comprise at least one of, an active device and a passive device (Duncombe, 
Figure 1 item 90, Figure 2 item 90, Jain, Figure 2) The examiner interprets that the 
display apparatus is a passive device. 

22. In regards to claim 1 7, Jain in view of Duncombe states the one or more viewers 
comprise at least one of, an intelligent device and a non-intelligent device (Duncombe 
Figure 1 item 90, Figure 2 item 90, Jain Figure 2). The examiner interprets that display 
apparatus is a non-intelligent device. 
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23. In regards to claim 18, Jain in view of Duncombe discloses the feedback 
comprises at least one of, a touch input, a typed input, a mouse input, a voice input and 
a facial expression input concerning the viewed scene (Duncombe, col. 6 lines 53-60) 

[the user 10 is an interface program 73 operably installed to 
function on a personal computer having a memory 70A and a 
processor 70B, as shown in FIG. 1. The personal computer 70 is 
operably connected to a data input mechanism 72, such as a 
keyboard 72 and a mouse 72A.] . 

24. In regards to claim 19, Jain in view of Duncombe states the feedback concerns a 
current scene and where the feedback comprises at least one of, a command to skip 
ahead in the playlist, a command to skip back in the playlist, a command to generate a 
new playlist, a command to find scenes similar to the current scene and a command to 
play a longer scene related to the current scene (Jain, col. 4 lines 27-30) [a panel 
172 displays the live video being digitized, with play, stop, 
etc. controls that interact remotely with the analog source via 

a deck controller 240 (FIG. 3) .] . 

25. In regards to claim 20, Jain in view of Duncombe discloses the playlist updater 
adds and/or removes a scene from the playlist based on at least one of, a usage data, a 
feedback command and a time stamp (Duncombe, Figure 2 item 1 12, or Figure 13). 

26. Claims are rejected under 35 U.S.C. 103(a) as being unpatentable over Jain et al 
(U.S. Patent Number 6,567,980) in view of Duncombe et al (U.S. Patent Number 
6,792,573). 
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27. In regards to claim 30, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. In addition, Jain teaches an annotating tool that 
facilitates creating the annotating metadata and associating the annotating metadata 
with the video segments (col. 8 lines 60-65) [During metadata capture, the 
user may mark video clips and annotate them. This input 552 is 
captured by the GUI Input Capture element 550. ]. 

The difference between the claims and Jain is the claims recite "a media 
database comprising a playlist and one or more video segments, where the video 
segments are associated with an annotating metadata" 

Duncombe teaches a system that allows the user to browser through video clips 
by receiving user feedback, which is similar to that of Jain. In addition, Duncombe 
teaches a media database comprising a playlist and one or more video segments, 
where the video segments are associated with an annotating metadata (col. 2 lines 34- 
36 & Figure 3B) [The method includes the steps of first organizing 
and formatting a plurality of media clips,] . 

It would have been obvious to one of ordinary skill in the art, having the 
teachings of Jain and Duncombe before him at the time the invention was made, to 
modify the viewing of GUI taught by Jain to include the database to create a play list of 
media files of Duncombe, in order to obtain a system that has the functionality of 
querying a database to retrieve digital media segments by user preference, and allow 
the user to record description information of the digital media. 



Application/Control Number: 10/055,538 Page 21 

Art Unit: 2173 

One would have been motivated to make such a combination because it create a 
dynamic search engine for viewing and organizing digital media information would have 
been obtained, as taught by Duncombe. 

28. In regards to claim 31 , Jain in view of Duncombe discloses the annotating 
metadata comprises at least one of, a date identifier, a time identifier, a videographer 
identifier, a face identifier, an item identifier, a voice identifier, a mood identifier and a 
theme identifier (col. 2 lines 16-20) [Using advanced media analysis 
algorithms that automatically watch, listen to and read a video 
stream, the multimedia cataloger intelligently extracts 
metadata-keyframes, time codes, textual information and an audio 
profile from the video in real-time. ]. 

29 . In regards to claim 32, Jain in view of Duncombe states the annotating tool 
generates the annotating metadata in response to a user input (Duncombe, col. 2 lines 
35-38) [The method includes the steps of first organizing and 
formatting a plurality of media clips, and then playing the 
plurality of media clips based upon input supplied by the 

user . ] . 

30. In regards to claim 33, Jain in view of Duncombe discloses the annotating tool 
automatically generates the annotating metadata based, at least in part, on a face 
recognition, an item recognition data, a voice recognition data, a mood data and a 
theme data (Jain, col. 6 lines 56-60) [Likewise, the remaining metadata 
tracks (Audio Classes 324, Speech 326, Speaker ID 328, Keywords 
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330) are each a parcel of metadata spanning a time period, and 
are extracted by their corresponding feature extractor shown in 
FIG. 9. ]. 

31 . In regards to claim 34, Jain in view of Duncombe discloses the annotating tool is 
adapted based on a machine learning technique based, at least in part, on a user input 
concerning the annotating metadata generated by the annotating tool (Duncombe, 
Figure 8A, Figure 3). 

32. In regards to claim 35, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. In addition, Jain teaches a media data store 
comprising one or more metadata annotated, displayable items (Figure 2 item 178); a 
presenter that presents a selected first displayable item from the media data store (col. 
4 lines 25-30) [Keyframes extracted during the capture process are 
displayed in a panel 176, while the corresponding close-caption 
text and timecodes are displayed in a panel 178.]. 

The difference between the claims and Jain is the claims recite "a selector that 
selects a second displayable item from the media data store based, at least in part, on a 
relationship between a first metadata associated with the first displayed item and a 
second metadata associated with the second displayable item." 

Duncombe teaches a system that allows the user to browser through video clips 
by receiving user feedback, which is similar to that of Jain. In addition, Duncombe 
teaches a selector that selects a second displayable item from the media data store 
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based, at least in part, on a relationship between a first metadata associated with the 
first displayed item and a second metadata associated with the second displayable item 
(col. 2 lines 65-67) [A further objective is to provide a media system 
that solicits feedback from the user and modifies the media 
presentation in response to the feedback.]. The examiner interprets the 
system takes the first parameter by the user, and then takes the second parameter, 
which is the feedback, to display a second metadata associated with the second 
parameter. 

It would have been obvious to one of ordinary skill in the art, having the 
teachings of Jain and Duncombe before him at the time the invention was made, to 
modify the viewing of GUI taught by Jain to include the user to manual search media file 
of Duncombe, in order to obtain a system that has the functionality of querying a 
database to retrieve digital media segments by user preference, and allow the user to 
record description information of the digital media. 

One would have been motivated to make such a combination because it create a 
dynamic search engine for viewing digital media information would have been obtained, 
as taught by Duncombe. 

33. In regards to claim 36, Jain in view of Duncombe states the media data store is 
at least one of a database, a data cube, a list, an array, a tree and a file (Jain, col. 3 
lines 60-67) [Metadata Server 130: may be as simple as a file system 
containing hypertext markup language (HTML) files, or as complex 
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as a relational database supporting a client-server application 
environment for media management. ]. 

34. In regards to claim 37, Jain in view of Duncombe discloses the presenter is at 
least one of an intelligent display and a non-intelligent display (Duncombe Figure 1 item 
90, Figure 2 item 90, Jain Figure 2). The examiner interprets that display apparatus is a 
non-intelligent device. 

35. In regards to claim 38, Jain in view of Duncombe discloses the presenter is at 
least one of an active display and a passive display (Duncombe, Figure 1 item 90, 
Figure 2 item 90, Jain, Figure 2) The examiner interprets that the display apparatus is a 
passive device. 

36. In regards to claim 39, Jain in view of Duncombe states the selector selects a 
second displayable item based, at least in part, in response to a user response to the 
first displayable item (col. 3 lines 43-45) [fig. 7 is a flow diagram 
illustrating the steps used to solicit, receive, and use 
feedback from the user to reselect the plurality of media clips;]. 

37. In regards to claim 40, Jain in view of Duncombe discloses the user response is 
at least one of, a spoken word, a keystroke, a mouse click, and a facial expression 
(Duncombe, col. 6 lines 53-60) [the user 10 is an interface program 73 
operably installed to function on a personal computer having a 
memory 70A and a processor 70B, as shown in FIG. 1. The 
personal computer 70 is operably connected to a data input 
mechanism 72, such as a keyboard 72 and a mouse 72A.]. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to O'Neal R Mistry whose telephone number is (571) 272- 
4052. The examiner can normally be reached on 9am - 6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W Cabeca can be reached on (571) 272-4048. The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). A / 
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